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%^ (54) Title: MANAGEMENT AND DELIVERY OF ONLINE WEBCASTS 

(57) Abstract: A streaming media deUvery system employs multiple client data netwoiks storing copies of the streaming media. 
^ When a viewer wishes to see content, the system chooses a client data network which is best for providing the media content and 
JlJ directs a client wrapper object on the viewer's system to that network. When the quality of delivery from that network becomes too 
^ low, a client wrapper object on the viewer's system can request a switchover to a new client data network. The system may also 
O redirect the wrapper object to receive content from a different network for maintenance purposes and the like. The viewer wrapper 
5^ object also provides monitoring information on communication line quality and the like to the system for feedback and logging 

purposes. 
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F1GUBK 1(a) shovvs an exan>ple of content streaming architecture/system 
and network according toai^eferred embodiment of the i«es«^ 

FiGUKE 1(b) depicts aspects of the content acquisition, publishing, 
distribution and deUvery system according to the present invention. 

PtGtJRES2(a) and2(b)aieblockdiagramsofacUentcomp«teranda 
network server, respectively, in accordance with embodiments of the present 
iiivention; 

FiGxmE 3 is a flowchart summarizing the process of a client obtaining a 

reference file according to embodiments of the present invention; 

RGirnE 4 is a state diagram of a client according to embodiments of the 

present invenlion; 

FIGUBE 5 is a flov^chart depicting operation of the redirection and 
bandwidth managers according to embodiments of the present invention 

FIGURE 6 is a state diagram of a monitoring manager according to 

embodiments of the present invention; 

IIGUBE 7 is depicts operations of the monitoring manager according to 

embodiments of the present invention; 

FIGIJRES 8(a) and 8(b) show the inclusion of advertisement in different 

types of streaming content deUvery according to embodiments of the present 
invention; and 

FIGURE9 isaflowchartoftypicaloperatiohsofsystems'accordingto 
embodiments of this invention. 

DETAIOiD DESCWPHON OF PBESENTXY PREFERRED EXEMPI^Y 

EMBODIMENTS 

PtGURE 1(a) shows a content stre^ning architecture/system and network, 
generally designated by the reference numeral 100. according to a preferred 
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e,ri«amentof thepresentinveotion. Amm*erofn=two*oU«.ts 102-1. 102-2, 
.... 102-n (coUeotivdy, iKtw* cHcnB 102) are coimecJed to 4^ 
dilributed «.«^ ne^™* kno«n as 4e Internet 104 via th«r «sp«:tive 
Meme. Service Provider (ISPa) 10<i (only o» ISP is sho™ in the drawing) in a 
manner knovminthe art. In the embodim«rtsal«.vm. the system 100 is 
in^emen.edover«repubach..en«tl04.usingtheWorldWidcWeb(WWWor 

Web) and its hyperlinldng capabiHties. The invention is wUcable to other 
^rks as although it is particularly snited to netv«».cs that are config^ 
.odlowh,perUnk..h>addition.««inventionisaIscappUcablctodo,™loading. 

10 wireless streaming and so on. 

BIGURE 2(a) shov^ an example of a cUent computer 102. Various types 

of network cUents 102 can be utilized, such as palmtop computers, notebook 
computers, personal organizers, and the like. CUent computer 102 includes 

conventional components, including a data processor, central processing unit 
,s (C3>U)108;volatileandnon.volatileprimaryelectronicmemoryllO;^^^ 
memory 112 sucb as hard disks, floppy disks and/or other removable media; 
network interface components 114; display devices interfeces and drivers 146; 
audio recording and rendering components 118; and other components as are 
commoninpersonalcomputers. Generally, the invention operates with / on any 
portable devices, inlcuding any device that can download and play content. 

Network cUents 102 are prefer^ly configured with a consumer-oriented 
operating system such as one of Microsoft Corporation's Wmdows operating 
systems, referenced inFiGURE 2(a) bynumeral 120. hi addition, network client 
102 runs an hxtemet browser 122 such as Netscape^M Communicator or 
25 Microsoft's Internet E3q>lorer. 
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Netwo* cite* 102 ito includes one or more stre^ring multunedia 
playersorrenderingcompon^^sm. Uris soft««e components) is capable of 
««*Bsldngs..e<.ningd...coonecttonsv*l>In.eme.serv«3 0roaers^^ 

of rendering the s««ning data ss audio or video. Sp«=ificany. player 124 is 
, eonfigured.oop«.ref«enceffles(describ«ibelow)andofes.ablisln.gs.r.an^ 
aa..conn.c.ions™tl.tl»ne,wodcresourcessp.cifiedinthereferen«ffles.using 

fl,e network transport protocols specified in ^ reference fUes. or in cot^nnction 
«ifl, fl» nrference file. -fte reference files are s.or«i on computer-teadable 
^ media of ser^rs or otl»r network so^^es or «e generated, as described 
„ below. Inp«ferred«nbodimentsofthisinvention.thepl.j«124isp»ferably 

,<,,,flg^«op«..eferencefllcssuchasASXflles(fileshavingaffl 
ext«nrton of -^-O! or ASF files (files 1-rving a filename extension of "asfO. 

Pl.yer(s) 124 can be implemented as a standalone component, program or 
a component control such as OCX control (OCX controls are standard feamres 

« of programs design^lfcr Windows operating systems), in dfter case, U is 

„gis,«ed wid. the operaang system so it is invoked to ipen the asp«>priate 

„*rencefiles(ASXfllesandfhelik.)hirespons.touserre,nests. tothe 
Microsoft Windows operating system, such a user request is made by double 
elickingonaniconrepresentingareferencefile.Fromwithinan,n,^browser, 
„ tt>ere<,u.s.fbra.eferenceflleism«l.bysel.cting(e.g..single^ckingon)a 
hyperlinkcontainedinahyperlinkdocumentthatisbeingdisplayed. Whentins 

happens, the player is loaded and executed, and the subj«* reference file is 
provided to tiie player as a run-time argument 

Each cUent 102 is or includes some fbm> of computer or shnilsr device 
« .spableofaccessingthedis.ributedcompu.ernetworkknownasthelnten>etl04 
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via ft.oU««-,re.p«tfv.ISP10«««»g..ypW AcUeotlOZmay 
aUo be adcdic^ed Wm».«nniml. . device «>cessingfl«I-^ 
«,e™io»sa«Klthelike. CUeo« 102 may have type of «»e««.«..ir 
,en«=ti« ISPs 106 using net«o* irterftc. 114. E.g.. cUe-Os may conneot to 
, ti«irISPsviamodems.otone««>rks,cabletdev«ons,slen«.ndft.like. 

E«hISP106isooanec»dinsomem.nner«.teIntemetl04.asistoovmin4e 

art. 

la some of the foBowing description, various euliti.s such as lie ISPs 106 
a„d««ln.««.104a«omi««l««»nda.aflowisdescribeddirectlybetween 
fl,evariouse«ities.Hov«v.r,,h.omissiouof««i.i.ssuchas.heISPs.ndthe 
in tl>e descripdon are merely to aid and simpliiy e^cplauatiou of fl» 

3ya«nl00. Actuddataflowb«ween««i«.s.e.g...clientl02-landthe 
^„nm.nag.rmv«»ldt,k.phcviathecUe.fsISP.06andtheIntemet 

104. 

Also connected to the Internet 104 is a «d> host 126 serving a v«b site 
128 to users over the interne. 104. CUents- 102 may iiew 40 con«« of *e web 

site 12S using i««n«t 1» " 

example, a client 102 m^ view the v«b site 128 using aNetsc^™ browser 
„^„nthecUent Tte web sit. 128 may offer, for e»nnple, straining m«ila 
content which &e cUent may access using the preferred embodiment. 

The content s.r«uning an*itecmre,system 100 also Includes various server 
computers. An «tample of a server computer is iUus.ra.ed in block Smn in 
FIOURE 2(b). Genially, a server computer includes convendonal component 
3i^«> those of networicdientm such .s a data processor (CPU) BO; volatile 
« ,„dno..vola«leprim»yelectronicmemoryl32:secondarymemoryl34such.s 
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hard disks and floppy disks or other removable media; network interface 
components 136; display devices interfeces and drivers 138; and othec 
components that are weU known. The server computer runs an operating system 
140 such as the Windows NT operating system from Microsoft Corporation. 
5 Network servers and their operating systMns are configured in accordance 

with known technology so that they are capable of streaming data connections 
with cUents. The network servers generaUy support various network transport 
protocols, including multicast UDP/EP and unicast protocols such as UDP/IP, 
TCP/IP, and HTTP. The servers include storage components (such as secondary 
10 memory 134), on which various data files are stored, formatted ^ropriately for 
efficient transmission using the noted protocols. Compression techniques are 
preferably used to make *e most efficient use of Ihnited Internet bandwidth. 

In the case of both network servers and cUent computer, the data 
processors are programmed by means of instructions stored at different times in 
15 the various computer-readable storage media of &e computers. Programs are 

typically distributed, for example, on floppy disks or CD-ROMs. The programs 
may be compiled or interpreted or, if appropriate, distributed and/or executed in 
some other manner. From the distribution media, the programs are installed or 
loaded into the secondary memory of a computer. At execution, they are loaded 
20 at least partially into the computer's primary electronic memory. Hie invention 

described herein includes these various types of computer-readable storage media 
v^en such media contain instructions or programs for unplementing the described 
steps in conjunction vdth a microprocessor or other data processor. Hie mvention 
also includes the computer itself when programmed according to the methods and 
25 techniques described below. 
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For pun>«ses of fflmtration only, progn^s. prog«m con^n«tfs ^ 

c„„p»«.althoughiti,«cogni»dtta.3uchprognuns.oom,»n«^ 
, „.ehm8n>s««de..variomtimestodi£fer«*s««^ 
computer. 

The oon«o. s«»ning «chitec««/s,^ IM acoordu^ « c""!"^ 
of P^sen. in«otion also includes a. leas. oa. «di«cdo. s«vor 142 ha^ a 

.«B«cSon -mmager 1« and a. one momU»i« s«ver 144 having a 
,„ „oni»nngnu„.agerl45.11.e«di«cdonscrv.rl42(alsore.^»b«in^ 
te«director)andd>emom«>ringserverl44arebo«.connec«d«,*eln»me. 
IM. tosome embodiment. *e«dix.ctionm.n...rl43«^moni»nngn«nager 
145mayl«coUoe.t«lonthesameservcrorfl»ymay*.ondifl^serve,s. 
Tt«ia.insomeeasesthexedi.ecttonserv.rl42andU«n«mi.oringserverl44 

,5 may be Hie same physical swvet. 

to ^Idition. «be content stteaminga«M»cto«/sys«m 100 accord^ 

p^embodin«n«ofthep«sen.invendonalsoincludesam^ofcon«nt 

dis«ibnttonne«.o*s (CDHs) 146-1, 14«-2 14«-k(coDecavely .eferredu. as 

CDNS146). h.p«ferr»dembodiMen«.som.crallof«.eCDNsl46 preferably 
,„ld*esames«amingmediacon»ntso.s«,provideadegreeof*«d.»..ran=e 

^aa^redundancyto^esys^m. ■n>en«nnerinv*ichda,a(con«nt)is 

,„^«>d disrtbu«d.o CDNs 146 «iU be described below reference U> 

aie content storage and disBibution system 148. 

Ape^onalization server 150, cUen. database 152. real^e b«>dwid.h 
^ ais.ribu.iondatabaselS4.CDNdat.basel56andanadver,isingdatabasel58are 
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also accessible to the redirection manager 143 and to the monitoring manager 145. 
either directly or via other servers. Ihe personalization server 150 accesses the 
client database 152 which contains information about particular clients and is 
used. e.g.. by the redirection manager 143 to customize a particular cUenfs 
i access. The advertising database 158 contains available advertising content which 

can be inserted into the content selected by the cUent The manner in which the 
advertising database 158 is populated by an advertismg storage and distribution 
system is described below. The monitoring manager 145 also has access to a real- 
time logging database 160. 

FIGURE 1(b) depicts aspects of the content acquisition, pubUshing. 
distribution and delivery system according to the present invention. As shown in 
Figure 1(b). various content providers including Fox 168-1, CBS 168-2, NBC 
168-3, ...» and PBS 168-q (collectively 168) provide streaming content in the 
form ofscheduled broadcast events and video on demand events. The content 
15 schedule and otiier information is stored in a electronic program guide 
(EPG) database 164 in a known manner. 

hi addition, a content database 170 contains the actual content obtained in 
various ways from the content providers 168. For example, the content database 
170 may be populated using a so-called crawler 172 which, along with channel 
20 definition mformation obtained from the EPG database 164. automatically 

uploads content to tiie content database 170. Further, content providers 168 can, 
if they so choose, push tiieir content to tiie content database 170 using content 
pushing. A storage manager 174 handles requests for storage and retrieval of 
content data to/from the content database 170. 
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In some preferred embodiments of the present invention, advertising is 
inserted in or included with streamed content Hie adverts are stored in and taken 
from an advertising database 158. An ad manager 174 takes ads ftom the 
advertising database ISSforinclusioninaparticularstream. In some preferred 
embodiments, advertising is included in the advertising database 158 based on a 
real-time advertising auctioa Various advertising agencies or advertisers (e.g., 
DoubleCUck 176-1. Engage 176-2, and other conventional/traditional broadcast 
advertising agencies, collectively ad. agencies 176) provide advertising to the 
advertising database 158. in some cases via an advertising auction, preferably a 

10 real-time auction. 

In the auction, various of the ad agencies 176 bid on advertising spots at a 
real-time ad auction site. Each advertising agent 176 maintains an account with 
the auction site and maintains credit (stored in the advertiser / finance database 
178) The auction site obtains the available time spot information from the EPG 

15 database 164. 

Once bidding starts for a particiolar time spot, the highest bids per spot are 
noted. Bidding for a particular time spot is closed before air time, preferably two 
(2) minutes before air time. For each bid, a check is made at the advertiser / 
finance database 178 to determine whether or not the bidder has sufficient funds 
20 or credit to pay for the bid. Ad agencies 176 can purchase additional credits on- 
line while they are bidding. 

WhUe the auction is taking place, all bids are ranked and all bidding 
parties are shown a count down to the end of bidding and the current bids. 
When the bidding closes, the ad with the highest bid is stored m the 
25 advertisingdatabasel58foruseatAe^ropriatetime. When the ad is included 
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in a reference file (and preferably vdien actually played), the cost of the ad is 
automaticaUy deducted ftom the winning agency's available credit in the 
advertiser / finance database 178. 

The operation of the system lOO is described now with reference to 

5 FiGtmES l(a)-6. 

Content is sent ftom the content database 170 to the various CDNs 148. 
based on various distributionrulesandprocedures. Multiple CDNs 148 are used 
to achieve data redundancy. Ibis means that data (content) are stored in more 
than one CDN distribution point, thereby providing fault tolerance. 

,0 insomepreferredembodiments,insteadofstoringthesameinformationat 

all CDNs 148. the information is stored at only some (e.g.. 20% to 40% of the 
CDNs). E.g.,ifthereareatotalofl0orl2CDNs.dataaiestoredat2to4CDNs. 
•nns approach provides redundancy whUe minimizing the cost of storage at 

CDNs: CDN storage cost is a factor ^pUed in load balancing of the system. 
,6 In preferred embodiments, twenty percent ofthe content in the content 

database 170 is stored to serve 80% of the traffic. 

In operation, when a cUent 102 chooses to view streaming media content 
from the web site 128, the cUent uses browser 122 to display Ihe contents of the 
web site 110 on some form of display (not shown) such as a computer monitor 
using display components 138. Each program or event which a cUent might wish 
to play (view, hear, etc) has a corresponding associated identity (Programid or 
Eve«ad, respectively). Also, each client (or user) has a corresponding associated 
identity {CUentld). When cUent requests specific content from the web site 128. 
the cUent's Internet brx,wser 122 provides both the cUent's identity (Clienad) and 
25 the requested program or event identity (Programid or Evendd). 



11 



10 



15 



PCTAJSOl/16329 

WO 01^1417 

In some aspects, the piesent invention incorporates an object mapper 
(monitor) program (described below). In preferred embodiments of the present 
invention, a cUent 102 needs an object vyrapper (morutor) program to be loaded 
onto the client 102. Accordingly, the first time a particular cUent 102 accesses the 
web site 128 in order to display streaming media content, a client object wrapper 
(morntor) 162 is downloaded automatically to the client 102. Tlxe cUent object 
tapper modulel62 may be downloaded usingawell-knownHIML technique 

such as the followii^: 

^nnnsCT classid=^LSID:931EFE2D-lF50-llD4-A821- 
id=wmpClieiitai style-^LEFT: Opx: TOP: Opx 

p7CDK=0&I^_NAME-bats-3w-filinchpl.asf&.asx'><OBJECT>. 

Once the wrapper module 162 has been downloaded onto the cUent 102. 
and when the vsreb page from the web site 128 is loaded into cUent' s system, the 

cUent' s browser 122 is redirected to the redirection manager 143 on the 
redirection server 142: 'ibe cUent's browser 122 provides the redirection manager 
143 with information allowing the redirection manager to redirect the cUent's 
request appropriately, according to the present invention. In preferred 
embodiments of this invention, the browser 122 provides the redirection manager 
143 with the both the cUent's identity iCUentId) and the requested program or 
event identity (Frogramld or Evenad). THat is. when the cUent 102 selects (e.g.. 
cUcks on) a link on web site 110 in order to obtain streaming content (e.g.. audio, 
video, download) therefrom, the cUent's request goes to redirection manager 143. 
m redirection manager 143 chooses (in a mamier described below) a particular 
CDN 146 to serve that requested media content to the requesting cUent 102. Once 

12 
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theredirectionn^erWatosdeterrninedvvMchp^^ 

102 is to use, the ledixection manager 143 then returns information back to the 
cUent 102 so that the cUent can immediately view the requested content The 

retuxnedinformationfromtheredirectionmanagerWSispreferablyaref^^ 

file. e.g.. an ASF (Advanced / Active Streaming Format). ASX. RM. SMIL 
(SynchronizedMultimediahrtegrationI^ge)<»rothertypeofcon^ In 
apreferred embodiment, the reference file sent from the redirection manager 

to the cUent 102 is an ASX file. For the sake of explanation only, and without 
loss of generaHty.the file returned by the redirection managerl43 to the che^^ 

,0 102 will be referred to herein interchangeably as the reference file and as the ASX 

file. 

A reference file is a text file that links Web pages to streaming media- 
based content on a media server. TlxereferencefilegeneraUyredirectsstreaming 
n.edia content away from the cUent's browser 103 to the appropriate media player 
« nxechanismmnmningonthecUent. In addition, in the pres^it invention., the 
reference file also redirects the cUent's media player mechanism 105 to the 
appropriate selected CDN 146. When the cUent's browser 103 downloads a 
reference file, the type of file (determine e.g.. from the file's name extension) 
causes the browser 103 to invoke the appropriate media player mechanism 105 
20 which then locates and plays the content specified in the file. 

Reference files are generally integrated into the WWW. Hyperlinks to the 
reference files are placed in Web documents, and a user retrieves a particular 
reference file by clicking on its hyperlink. In response, the user's Internet browser 
122 retrieves the reference file from the network source and opens it with the 
.s appropriateplayerl24.dependingon1hetypeofreferencefile. Playerl24.in 
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tun^ uses the reference file to establish a streaming data connection which the 

player then renders. 

A typical BfcKoce file tf^V ^""^ ' P^"^ of lines, each 
e<,n.dni»g a diflereat r«K»m» sp«ffi« in 

Locator (UW.) format The order of tl.e resowc. specifiers establishes . 
p,efe„ed o«ler fbr a.t«.*.ing commumcadons «i.h ie resources specified by the 
source specifiers. Each resource sp»ifier is preceded, in some pn-ocol.. by 
identifier of the form -Re«-URI.". where the # portion of the idenafier is a 

n^ber ^ch indic«es *e preferred otder for attempting communications. For 
0 example.iie«isbeforei!e/2.Altematively..heref^cefllecanspecie,the 

otder by referencing »K,.her file that in turn conBins a specification of 

resources in their preferred order. 

EachresourcespeciflerdeslEnat.sanetwo*resourceandap«>tocol 

specifier. A ptaraUty of different resource spedfiets can be pUced on diffisrent 
„ Itaesofanrf^encefile. Whenplayermopensandreadsarefcrencefile.it 
responds by repeatedly attempting to estd^lish a stresnnng data com«^ 

the difieren. resource ,p«=ifiers m prefeoed order specified by ^ refer«»>e 
file mitil a streaming data connection is successfijlly established. 

Abasicrefcrence(ASX)filecontainsatleasttheUIU,ofsome 
„ n,uWmediacont«>tonaser»er.Acomplexlh*filemaycontainmuWplefll^ 

s.reamsarrangedinaplayUst.ins.r.ctionsonhowtcplaythefllesorstreams.text 

and graphic el«nents. and hyperlinks associate! vdth elements on fte media 

player interface. 

An example of an ASX file is show in TABLE I below. ^ 



TABLE I — Sample ASX File_ 
<ASX version =^'3.0''> 
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TABLE I — Sample ASX File 



<Title>Titan AE</Title> 



<Abstract>Ex flm pIe caU for Broadstream customer5</Abstract> 



<MoreInfo href ° nittp'7/www.microsoftcom"/> 



<Entry> 



<Title>Titan AE<nritle>_ 



<Aiithoi>www.reelstream.coin</Autfaor> 



<Copvrigh1>1999, Paramoxmt Pictures ^ Inc,</Copyrigtit> 



<Abstract>preview from Titan A£<;/Abstract> 



10 



<MnreTiifo href ^ "http'7/wwwjeelpreviews.com/^'/> 



11 



href^''http'7/fusion,broadstreamxoro/reelpre view/Ca^ AE.asf-/> 
</Entry> — 



12 
13 



<Eiitry> 



14 
15 



<Title>Titan AE</Title> 



<Authoi>www.reelstream.com</Author> 



16 



<rnnv right>2000. Paramount Pictures . Inc</Copyright> 



17 



< A hctrflcoendins credit for Titan AE</Abstract> 



18 



<MoreInfo href "http://www,reelpreviews.com/"/> 



19 



20 
21 



■^Rcf 

href='WV/&5ion.broadstream.coin/reelpreview/Calliiig.^^ 

sf"^^ . ■ 



</Entri» 



</ASX> 
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The various elements of ASX files are known in the art and can Be found, 

e.g., in 

http://msdn.mic^so£tcommbrary/psdk/vmi_media/vvmpsdk/nmp_s^^ 
ASX_mtro.htm, vdiich is incorporated herein by reference. However, some 
elements are explained here for convenience. 

Jh& Entry ("<ENTRY>") elanent of the file (e.g., hnes 5 andl3 in Table 
I), with its associated attributes, defines to the player mechanism 124 meta- 
information for a single, logical piece of content (called a cUp). Elements that are 
defined within an Entry element are displayed by the player mechanism 124 in a 
particular information area of the display panel called the CUp information area. 
A playUst is created by stacking multiple entries. Each Entry element is played 
by the player mechanism 124 in the order they appear jn the file as though the 
user had manvially opened each clip. 
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The Jl^elemeiit (e.g., lines 1 1 and 19 in Table I) specifies a URL for a 
content stream. The URL can point to any supported media type, using any 
protocol supported by the player mechanism 124. The i?e/element is commonly 
used for server or protocol rollover, where, if the player mechanism 124 cannot 
access media defined in a Uc/element, it tries to access the URL in the next Ref 
element. 

Preferably the ASX file produced by the redirection manager 143 is 
customized for the particular requesting cUent 102 based on cUent information 
v*ich the redkection manager 143 either has or can obtain from the client 102. 

Amongst other things, the returned ASX file points the cUent's browser 
103 to the requested streaming media content on one of the CDNs 146 . As noted 
above, some or all of the CDNs 146 hold the same streaming media content to 
provide a degree of feult tolerance and data redundancy to the system. In some 
embodiments, only some of the CDNs 146 will hold the same content The ASX 
file may contain pointers, e.g., in the form of URL's, to more than one content 

The process of a client 102 obtaining a reference file is summarized with 
reference to Figure 3. First, the CUent 102 (Clientld) selects streaming content 
(Programld) from Web Site 110 (at 300) by selecting a hyperlink on the web site. 
The CUent 102 is redirected to the Redirection Manager 143 at the Redirection 
Server 142 (at 302). If needed (e.g., this is a first-time cUent), the cUent object 
wrapper (monitor) module 162 downloaded automaticaUy to the cUent 102 (at 
304). The redirection manager 143 then passes the request for the event to a 
reference file generator (at 306) which generates a reference file based on certain 
information including the Clienad (to obtain information from the CUerit 
Database 152 direcdy or via the personaUzation server 150), the Programld and 
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avaUable program events (ftom Progran Event database 164). advertising (ftom 
Advertising Database 158) and avaUable. preferably best available. CDNs 146 
(provided by the band>vidth manager 166) (at 308). Then the generated reference 
(e.g., AS30 file is returned to the client 102 (at 310). 

5 In prefeired embodiments of the present invention, the reference file 

includes a list of CDNs fiom which the content is avaiteble. 

Once the cUent 102 has received the reference file firom the redirection 
manager 143, the cUent's player mechanism 124 begins playing the content 
referred to by the reference file, preferably under the control of the wrapper 162. 

10 When the cUent system 102 accesses the streaming media content on one 

of the CDNs 146, the appropriate CDN 102 begins deUvering content to Ihc client 
102 as is known in the art and the cUent renders (e.g., plays and/or displays) the 

content using the appropriate (built-in or plugin) player mechanism 124. 

While the client 102 is receiving streaming media content &om the CDN 
15 146, the cUent's object wrapper 162 continuously monitors the received network 

traffic to evaluate quaUty of service (QOS) at the cUent 102 by measuring, e.g., 
network reUability, network congestion and network quality. In some preferred 
embodiments, the object wrapper 162 monitors the received network traffic for 
such measures as total bytes received, number of lost packets, number of 
20 recovered packets and the like. The cUent object wrapper 162 sends network 

status infonnation to the monitoring manager 14S at regular intervals. 

The monitoring manager 145 processes all infonnation received &om 
cUents. as described below and with reference to Figure 6. Some or all of the 
information is stored in various databases, including in the real-time logging 
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database 160, the cUent database 152. the CDN database 156, and the real-time 
bandwidth distribution database 154. 

In a presently preferred embodiments of this mvention. three types of 
. information are sent from the cUent 102 to the monitoring manager 145. namely 
network status information, user action information and so-called ping 
information. Along widi whatever infomiation is sent, the cUent object wrapper 
162 also transmits an indication (e.g., a tag) indicating to the monitoring manager 
145 what kind of data/request is bemg sent. 

Ite network status information is sent from the cUent to the monitoring 
manager 145 regularly, preferably at fixed intervals. In one preferred 
embodiment network status uifbmiation is sent every two (2) minutes, a^^ 
includes such infonnation as the number of bytes transferred, the number of 

packets sent and/or received and/or recovered, measures of transmission quaUty 
andothertypesofnetworkmformation. hx some embodiments, the network status 
infonnation includes the foUowing items: 
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ChentldCode 



Programld. 



hu fferin^omt 



hu fferimClient 



Clientid 



The unique identity of the client 



The unique identity of the program bemg view 
by the client 



Rate 



Current Position 
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When Ihe monitoring manager 145 receives a message/request from a 
cUent 120 (ftom the cUenfs wrapper 162) (at 700 in Figure 7), the monitoring 
manager determines -what type of message it has received (at 702) using the tag or 
code CmdCode sent with the message. If the message tag indicates that the 
message is network status information (CmdCode = 0x0001), the monitoring 
manager 145 determines that it has received real-time logging mformation which 
it stores in the real-time logging database 130 (at 704). 

The user action/event information sent to the monitoring manager 145 
includes data on acts performed by a user at the cUent 102. Whenever a user at 
the cUent 102 performs an event such as plays video, stop, rewind, fast forward 
and so forth, this user event is transmitted to the monitoring manager 145. In 
some preferred embodiments, the user action/event mformation includes: 



User Action/Event 
Information 


Meaning 


CmdCode » 0x0008 


Code indicating type of information for use by 
monitoring manager 


ClientidCode 




Programid 




StartTime 




EventCode 


Play (0x0001) 
Stop (0x0002) 
FastForward (0x0004) 
FastReverse (0x0008) 
Next (0x0010) 
Previous (0x0020) 
Pause (0x0040) 
Cancel (0x0080) 
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When the monitoring manager 145 receives user action / event information 
(CmdCode = 0x0008) from a client (at 700), it • . . (at 706). 

The monitoring manager 145 saves all information into the database. The 
saved information may be data mined at later time to generate various reports, 
e.g., for customers. 
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The ping information is smt to the monitoring manager 145 regularly, in 
some embodiments, preferably eveiy thirty (30) seconds. The ping information is 
used to allow the monitoring manager 145 to know that the client object is still 
connected and alive. 



Ping Information 


Meaning 


CmdCode = 0x0002 


Code indicating type of information for \ise by 
monitoring manager 


ClientidCode 




GetCttrrentEntryO 




CtarentPosition 





When the monitoring manager 145 receives a Ping mess£^e {CmdCode = 
0x0002) from a client 120 (from the client's wrapper 162) (at 700), the monitoring 
manager uses the ping information to map bandwidth distribution (at 708) and 
stores the associated information in the real-time bandwidth distribution database 
124 (at 710). 

If the client object monitor (object wrapper 162) detects a problem with 
network quality, it initiates a CDN switch-over with the monitoring manager 114 
in order to link to another CDN 146 for the content. 

In addition, the client object monitor 162 automatically restores services 
when streaming or downloading becomes interrupted because of a network 
problem. The client object monitor 162 sends a switch-over request message to 
the monitoring manager 114 requesting permission to switch to another CDN for 
uninteirupted download. A switch-over request message from the client object 
monitor 162 preferably includes the following: 



Switch-over Request 


Meaning 


CmdCode = 0x0004 


Code indicating type of information for use by 
monitoring manager 


ClientidCode 




Programld 




StartTime 
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Svrttch-over Request 


Meaiiiii£[ 


BaseURL CDN.ID 


the URL of the current CDNCDH ID is a 
number that is used internally to identify a CDN. 
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In response to a switch-over request fiwm a client 102 (CmdCode = 
0x0004), the monitoring manager 114 evaluates the real time network status and 
selects a new CDN based on such factors as availabiUty, reliability and cost- 
effectiveness. More specifically, the monitoring manager 145 first checks the 
current load balancing and distribution (at 712), and then generates a CDN 
switchover command/message (at 714). The identity of the new CDN is provided 
to the cUent 102 from the monitoring manager 114 in the form of a CDN switch- 
over r^ly message (at 716). 

A CDN switch-over reply message preferably includes the following: 



Switch-over Reply 


Meaning 


CmdCode = 0x8001 


Code indicating type of information for use by 
client wrapper object 


ClientldCode 




iSwitchOver (1/0) 




BaseURL CDNJDD 


the URL of the new CDN to be switched to CDN 
ID is a number used internally to identify a CDN. 
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Upon receipt of a CDN switch-over reply message fiom the monitoring 
manager 114, the client 102 attempts to switch to the new CDN (represented by 
the baseURL field of the switch-over reply message). 

After the client 102 switches over to the new CDN, tiie client wrapper 
object 162 sends a report to the monitoring manager 114 to that efifect. 

If the client wrapper object 162 is unable to connect to the monitoring 
manager 114 in order to mitiate a switch over, the client may initiate a self- 
controlled or self-directed CDN switchover. The reference file will preferably 
include a list of CDNs firom which the content is available. If a client needs to 
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make a self-directed CDN switchover, the client needs only to select one of the 
CDNs from the list in the reference file. 

Preferably a CDN switchover, whether client or manager initiated, causes 
little or no disruption to the content being displayed. However, since the content 
is being streamed to tiie client from the original CDN, the new CDN needs to 
begin its content streaming at or substantially close to the current position of the 
current streamed content 

When the client wrapper object 162 determines that the quality of the 
stream from the CDN 146 has degraded to an unacceptable level, e.g., there are 
too many lost packets or the like, the client wrapper object 162 preferably sends a 
CDN change request to the monitoring manager 114. Upon receiving such a 
request from a client system, the monitoring manager 114 chooses a new CDN 
146 to supply the streaming media content to the client as described above. The 
monitoring manager 114 then sends a message to the viewer system (the client 
102) identifying the newly-chosen CDN. The client 102 polls its media player to 
identify the streaming media which is being played, as well as a time index 
thereof, e.g., how far the playback is into the stream. The client 102 then 
advances this time index a predetermined amoimt of time and sends a request to 
the new CDN server asking it to begin sending a stream from tiie identified 
content to the client at that time. Also, at that advanced time, the client 102 
directs its streaming media player to terminate its stream with the old CDN. In 
this way, reception and playback of the streaming media continues virtually 
interrupted by the changeover to the new CDN. Currently, this predetermined 
time is around 7-8 seconds. That includes about two to three (2-3) seconds of 
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coiinection time and about five (5) seconds of buffering. These ti m i ng numbers 
assume good network conditions. 

A preferred way to do a seamless switch over (i.e., perceptibly 
instantaneous switchover) is as follows. A first Windows Media Object plays on 
the screen, and a second instance of the Windows media player object is 
instantiated behind the first one. Both Windows media player objects are 
synchronized by starting the second Windows Media Object and settmg its time 
mark (current entry and current position variable) equal to that of the first 
Windows Media Object After the second Windows Media Player finishes 
buffering and starts to stream, the first Windows Media Player is killed. Thus 
creating an effect of instantaneous CDN switch over. 

The monitoring manager 114 can also initiate a change in the CDN 146 
associated with a particular client 102. It may do so, for example, when 
scheduled maintenance is to be performed on one of the CDNs 146. At a 
predetermined tune in advance of the beginning of the maintenance period, the 
monitoring manager 114 sends commands to all clients 102 receiving streams 
fi-om that CDN directing them to begin receiving the streaming content fixjm 
selected other ones of the CDNs 146. To do so, the monitoring manager 114 Gxst 
selects new CDNs for each of the affected clients, sends CDN switchover 
commands to each such client advising it of its new CDN. The aflFected clients 
affect CDN changeovers as described above. Presentiy preferred embodiments 
vise HTTP instead of TCP/IP to communicate* with the monitor. This allows 
clients to communicate with the monitoring software through a standard HTTP 
port. This also means that the CDN switch over command can be issued during 
the PING cycle. 



23 



wo 01/91417 PCTAJSOl/16329 

In addition to being used for changeover purposes, logging information 
such as the above is sent to the monitoring manager 114 via the Internet 104. The 
monitoring manager 114 stores this logging information in a client information 
database 120. The monitoring manager 114 puts all real time network data into 
the database. It also saves history of network information in the database. The 
monitoring server can at anytime look at the status of the multiple CDN network 
and measure network quality and network traffic. The network vriXL adjust itself to 
balance itself as problems arises. Additionally, information stored in this database 
for a particular content provider is available for viewing in real time by that 
content provider via a web page on the web server. Preferably, system 
administrators can view all logging information as well as statistics derived 
therefrom. 

In summary, with reference to the state diagram in Figure 4, a particular 
client 102 is initially in a start state (S400). As described above, the client 102 
requests a reference file from the redirection manager 143. When the ASX file is 
retumed to the client 102, the client connects to the appropriate CDN 146 and the 
client's player mechanism 105 begins to play the streamed content (at state S402). 
While the streamed content is being played, the wrapper 162 monitors the system. 
If the wrapper deteraiines that the network has failed or otherwise determines that 
the network performance is unacceptable (e.g., there is network congestion), tiie 
wrapper 162 moves to a ''network bottleneck or CDN switchover"' state (S404). 
The wrapper 162 notifies the monitoring manager 145 about the problems and 
waits for instructions. While waiting for instructions from the monitoring 
manager 145, the client's player mechanism 105 continues to play the content, if 
possible (i.e., if there has not been complete network failure). 
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If the wrapper 162 does not receive instructions from the monitoring 
manner 145 within a predetemiined amount of time Q>referably less than X 5- 1 0 
seconds), or if the client's wrapper 162 cannot establish a link to the monitoring 
manager, the client initiates a self-controlled CDN switchover (in state S406). On 
the other hand, if the wrapper 162 receives timely instructions from the 
monitoring manager 145, the client initiates a monitor-controlled CDN switchover 
(in state S408). In either case, v^en the switchover has taken place, the client's 
system returns to the play state (S402). When the stream ends, the system enters 
the state. 

The Redirection & Bandwidth Managers 

Upon a client request.for streaming content (at 300 in Figure 3), the 
redirection manager 143 must detem^ne which CDN 146 the requesting client 
shovdd use. The decision by the redirection manager 143 as to v^ch CDN to use 
is made preferably according to predetermmed rules, e.g., weighing factors such 
as network load, reliability and cost effectiveness* 

The redirection server 142 includes a bandwidth manager mechanism 128 
which provides the redirection manager 143 with information needed to make its 
decisions. As shown in FIGURE 5, the redirection manager 143 periodically asks 
(at 500) the bandwidth manager mechanism 128 how to weight each CDN 146 per 
a particular content distributor. In some preferred embodiments, the redirection 
manager 143 asks the bandwidth manager mechanism 128 for information at least 
every ten (10) minutes. In some preferred embodiments, the redirection manager 
143 asks the bandwidth manager mechanism 128 for information when needed in 
addition to periodically. 
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in response to the request ficom the redirection manager 143 (or on an 
ongoing basis) the bandwidth manager mechanism 128 determines (at 502) how 
to balance the load, minimize the cost and maximize the performance of the 
Systran 100. Accordingly, the bandwidth manager mechanism 128 queries the 
cUent database 122 (at 504) to determine which CDNs 146 are working with this 
distrib\itor (customer). In addition, the bandwidth manager mechanism 128 . 
queries the CDN database 126 (at 506) to determine the cost and pay structure per 
CDN146. The bandwidth manager mechanism 128 also queries logging 
information (at 508) in the real-time logging database 130 to evaluate recent 
network reUability. Having obtained the available information, the bandwidth 
manager mechanism 128 calculates (at 510) how to balance the load, minimize 
the cost and maximize the perfonnance.of the system 100 and provides the 

redirection manager (at 512) with information needed to select an appropriate 
CDN 146 for a particular client and content selection. 

Once the redirection manager 143 has determined (at 514) which 
particular CDN 146 the cUent 102 is to use, the redirection manager 143 then 
returns information back to the cUent 102 so that the cUent can immediately view 
the requested content. 

Preferably the redirection manager 143 selects tiie lowest cost CDN 146 
for any particular client and content. 

Advertising 

In some embodiments of the present invention, when the redirection 
manager 143 creates the ASX file for a particular client, advertising information 
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(or other content) is added to the ASX file. The advertising may be directed to the 
client based on the client's identity (Clienad) and other information. 

As ASX file or the streamed content can include instructions to the player 
mechanism 105 to cut away firom a stream and to play other streams or files 
according to scripting in the file. For example, during a live Internet broadcast of 
an event, a script command can be sent at the beginning of every commercial 
break that instructs each client to play commercials listed in their respective ASX 
files. When clients finish playing the commercials, scripting in the metafile 
instructs each client to cut back to the live broadcast. . Advertising insertion is 
preferably implemented using the Event element. 

Figures 8(a) and 8(b) show the inclusion of advertisement in different 
types of streaming content delivery according to embodiments of the present 
invention. Specifically, FIGURE 8(a) shows the processing real-time ad insertion 
in regular (scheduled) programs, whereas FIGURE 8(b) shows that processing for 
i«al-time ad insertion for live events. 

With reference to Figure 8(a), when a client 102 requests streaming 
content (i.e., requests a reference file) that plays a scheduled event (at 800), the 
redirection manager returns to the client (at 802) a reference file that, when played 
by the client's player 124, will play that event. The client's web browser asks the 
ad manager 174 what stream is to be played with this event (at 804). In response 
to this request, the ad manager 174 queries and gets information firom the 
advertising database 158 and firom the personalization server 150 (at 806). Using 
this information, the ad manager 174 determines which ad is to be played by the 
client and downloads the ad stream information to the client's web browser (at 
808). The client's player 124 starts to queue the ad for play (at 810). When the 
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client's player reaches the next item on its play list (the ad), it plays the queued ad 
(at 812). Then, when the ad is done, the player 124 plays the next contrat item on 
itsplaylist(at814). 

With reference to FIGURE 8(b), when a client 102 requests live streaming 
content (i.e., requests a reference file) that plays a Uve event (at 816), the 
redirection manager returns to the client (at 818) a reference file that, when played 
by the client's player 124, will play the requested event. Unlike the case of 
scheduled events, where the playlist has markers for the insertion of ads, in the 
case of a live event, the ad manager issues an event call to the client when an ad is 
to be played (at 820). In response to such a call, the client asks tiie ad manager 
174 what stream is to be played with this event (at 822). In reg>onse to this 
request, the ad manager 174 queries and gets information Scorn the advertising 
database 158 and fix>m the personalization server 150 (at 824). Using this 
infomiation, the ad manager 174 determines which ad is to be played by the client 
and downloads the ad stream information to the client's Web browser (at 826). 
The client's player 124 queues the ad for play (at 828), stops playing the live 
event and then plays the queued ad (at 830). Then, when the ad is done, the 
player 124 continues with the live event (at 832). 

In both of the cases described above, the ad is preferably selected based on 
client personal) information, the ad spot and the content being viewed. 
Preferably an ad is ndt selected until the spot has been sold and the ad is then 
served in real-time with an Event call to the client. Normal programming is 
resumed after the advertisement call. 
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Example of Operation 

An example of typical operation of embodiments of this invention is 
presented here, by way only of example. 

Real Time Load Balancing With Real Time Feedback 

Figure 9(a) generally shows real time load balancing with real time 
feedback according to the preferred embodiment of the present invention. This 
example assumes that the client has used the system before and so does not need 
an initialization process. That is, in this example, it is assumed that the client 
wrapper/monitor mechanism 162 is present on the client's system. As shown in 
Figure 9(a), first a user/client requests a particular video (at 900). This is done, 
e.g., by tiie client selecting a hyperlink for that video on a particular web site. The 
hyperlink directs the client to the redirection manager 143 and the client provides 
the redirection manager 143 with the client's identity and the program identity for 
the desired video. The redirection manager 143 creates a reference (e.g. ASX) file 
for the requested video and returns the reference file to the client (at 902). The 
client's player 105 then begins to play the video (at 904) while, at the same time, 
the client wrapper 162 starts to monitor the system (at 906). If the wrapper 162 
determines that there is network congestion (or failure) (at 908), then the client 
contacts the monitoring manager 145 (at 910), reports the problem and asks for a 
monitor-controlled CDN switchover. The client then waits for a response from 
the monitoring manager 145. While waiting for a response from the monitoring 
manager 145, the player 124 continues playing, if possible C^-c, if the connection 
to the CDN has not been completely lost and/or the player 124 has some content 
buffered) - 
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If no netwoik congestion was detected (at 908) by the client wnq^per 162, 
then, if scheduled or necessary, the wrapper 162 reports any information (e.g., 
ping, status, etc.) and/or user events to the monitoring manager 145 (at 912) and 
processing continues. If all of the requested content referred to in the reference 
(ASX) file has been completed by the player 124 (at 914) then processing is done 
(at 916), otherwise playing and monitoring continue (at 904 and 906). 

If the client detects network congestioni (at 908) but is unable to contact 
the monitoring manager 145 or the wait for a request to switchover to another 
CDN times out (at 918), then the wrapper/monitor 162 initiates a self-controlled 
CDN switchover (at 920) and then processing continues (at 914), if there is any 
content remaining to be played. In the case of a self-controlled CDN switchover, 
the player 124 switches over to another CDN specified in the reference file. 

On the other hand, if the client determines that there is congestion (at 908) 
and is successfiil in contacting the monitoring manager 145 (at 910) and does not 
time out, the client obtains new CDN informatioirfixjm the monitoring manager 
145 (at 922) and initiates a CDN switchover (at 924) to the new CDN. If the 
program has been completed (at 914) then processing is done (at 916), otherwise 
playing and monitoring continue (at 904 and 906). 

The various mechanisms described herein, including, wi&out limitation, 
the monitoring manager, the redirection manager, the bandwidth manager and the 
wrapper / monitor mechanism may be implemented in hardware, isofhvare or a 
combination thereof. When implemented in software, they may be implemented 
in any type of appropriate interpreted or compiled programming language. In 
preferred embodiments of this invention, the wrapper mechanism is implemented 
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in the machine-independent Java™ programming language as small, powerful and 
fast C++ ATL COM objects. When implemented fully or partially in software, 
aspects of the invention can reside on any memory or storage medium, including 
but not limited to a ROM, a disk, an ASIC, a PROM and the like. 

When the various mechanisms of the present invention are running on a 
pardcnlar machine (e.g., the at the client or on a server), they may reside in the 
memory of the machine or on a stor^e device or in a combination. 

While the invention has been described vnih reference to particular 
mechanisms (algorithms, processes and functions) and architectures, one skilled 
in the art would realize that other mechanisms and/or architectures could be used 
while still achieving the invention. 

While embodiments of the present invention have been described with 
particular setup and initialization procedures, other setup and/or initialization 
procedures can be used. 

Further, while many of the op^tions have been shown as being 
performed in a particular order, one skilled in the art would realize that other 
orders, including some parallelization of operations, are possible and are 
considered to be within the scope of the invention. 

The present invention has been described above in coimection with a 
preferred embodiment thereof; however, this has been done for purposes of 
illustration only, and the invention is not so limited. Indeed, variations of the 
invention will be readily apparent to those skilled in the art. For example, the 
redirection and monitoring servers may be combined into a single server, or the 
system may additionally be used to store content developer's streaming media 
content. Such variations also fall within the scope of the invention. 
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What Is Claimed Is: 

1 . A system for delivering streaming media over a distributed 
conmrunication network comprising: 

a pluraKty of content distribution networks (CDNs) for delivering the 
streaming media to a plurality of clients; 

load balancing means for evenly distributing a delivery load imposed by 
the plurality of clients over the plurality of delivery means; and 

clieait wrapper means, in each client, for providing information to the load 
balancing means for use in distributing the delivery load. 

2. A system as in claim 1 wherein each client wrapper means 
comprises: 

means for monitoring network traffic received at the client to evaluate 
quality of service at the client 

3. A system as in claim 2 wherein the means for monitoring 
comprises: 

means for measuring at least one of network reliability, network 
congestion and network quality. 

4. A system as in claim 3 wherein the means for monitoring monitors 
the received network traffic for at least one of: (a) total bytes received, (b) number 
of lost packets, and (c) number of recovered packets. 
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5. A system as in claim 2 wherein the means for monitoring sends to 
the load balancing means at least one of (a) network status information; (b) user 
action information; and (c) client status information. 

6. A system as in claim 5 wherein the network status information is 
sent to the load balancing means regularly. 

7. A system as in claim 6 wherein the network status information is 
sent at least every minute 

8. A system as in claim 6 wherein the network status information 
includes at least some of: (a) a number of bytes transferred to the client, (b) a 
mmiber of packets sent and/or received by the client; and (c) a number of packets 
recovered by the client from multiple CDNs. 

9. A system as in claim 1 wherein the client wrapper means 
comprises: 

means for detecting a problem with network quality; and 
switchover means for, when a problem is detected with network quality, 
initiating a CDN switchover. 

A system as in claim 9 wherein the switchover means comprises: 
means for requesting a CDN switchover from the load balancing 



10. 



means; and 
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means for perfonning a self-controlled CDN switehover in the 
event that a requested switchover does not take place. 

11. A method of managing deUvery of streaming media in a system 
wherein streaming content is deUvered to a cUent from a pluraUty of content 
distribution networks (CDNs), the method comprising, at a cUent: 

obtaining a reference file containing a reference to at least one CDN; 
requesting streaming content from a CDN in the reference file; and 
monitoring traffic received at the cUent to evaluate quaUty of service at the 

cli^t. 

12. A method as in claim 1 1 further comprising: 

sending to a monitoring manager information regarding the 

monitored quality of service at the client. 

13 . A mefliod as in claim 12 wherein the information sent includes at 
least one of (a) network status information; (b) user action information; and 

(c) client status information. 

14. A method as in claim 1 1 further comprising: 

initiating a CDN switchover if a problem is detected with network quality, 

15. A method as in claim 1 4 further comprising: 

performing a self-controUed CDN switchover in the event that a requested 
switchover does not take place. 
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16. A method as in claim 1 1 fiirther comprising: 

receiving an instructiop to perforai a directed CDN switchover; and 
performing a directed CDN switchover. 

17. A method of managing delivery of streaming media in a system 
vrtierein streaming content is delivered to a plurality of clients from ones of a 
plurality of content distribution networks (CDNs), the method comprising, at a 
monitoring manager: 

receiving status information from ones of the plurality of clients; 
based on received status information, directing a particular client to 
switchover to a different CDN. 

18. A method of managing delivery of streaming media in a system 
vs^erein streaming content is delivered to a plurality of clients from ones of a 
plurality of content distribution networks (CDNs), the method comprising: 

receiving a request from a particular cheat of tiie plurality of clients to 
obtain a particular streaming content; 

selecting a particular CDN to provide the requested content to the 
particular client; and 

providing the client with a reference to the selected CDN. 

19. A method as in claim 1 8 wherein the selecting of the particular 
CDN is based on at least some of: 

(a) available CDNs; 
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(b) the requested content; 

(c) the status of the network; and 

(d) pricing information. 

20. A method as in claim 1 8 further comprising: 
providing the client with a list of CDNs from which the content may be 
obtained. 
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